In various areas of machine learning and statistics, mathematical models are constructed, learned, or derived for classifying data into categories, classes, or subsets. For example, machine learning may be used to generate models for categorizing handwritten characters as the corresponding alphanumeric character, recognizing a portion of a photo as a face, or the face of a specific person. In another example, machine learning may be used to generate models that predict tomorrow's temperature based on various current weather elements or predict a patient's prognosis based on the patient's symptoms. In yet another example, the models may classify whether a particular e-mail is a spam e-mail, based on the content of the e-mail.
Models classify data or make predictions that utilize the data based on a number of features associated with the data. The models include algorithms and rules to process the data and generate the classifications or predictions. Models are used in a wide range of applications, including analysis of genetic data, financial modeling, image classification, sorting tasks, weather prediction, and many other applications. By utilizing these models, data can be sorted to aid in extraction of useful information, including trends, from the data.
The accuracy of a model may vary depending on the number of features used by the model to classify the data. In some cases, there may be an optimal or improved set of features used by the model, wherein the improved set of features allows the model to classify the data more efficiently and accurately. In such situations, if the model utilizes a lesser number of features than that in the optimal set, the classification accuracy of the model may be compromised. On the other hand, if a model uses a greater number of features than that in the optimal set, the computational cost of the model may increase without a significant gain in accuracy. For instance, in the example of predicting a prognosis for a patient with a disease, a model may generate the prediction based on a feature set of one feature, such as the age of the patient. The model may be able to generate predictions with minimal computational resource, but the predictions may lack accuracy. In another situation, the model may generate the prediction based on a feature set of several hundred features, including age, weight, genetic profile, previous illnesses, previous treatments, etc. The model may be able to generate predictions with high accuracy, but the predictions may be computationally expensive. An optimal feature set would include enough features to enable the model to generate a classification or prediction with an acceptable level of accuracy, while excluding features that are not necessary for the desired level of accuracy.
There is a need for systems and methods that can determine the optimal feature set for various models.